R/Pharma Summit @ Posit Conf 2024, August 11th
Slides available: pharmar.github.io/events-positconf2024
On behalf of the R Validation Hub team:
Aaron Clark Arcus Biosciences
Doug Kelkhoff Roche
The R Validation Hub is a collaboration to support the adoption of R within a biopharmaceutical regulatory setting (pharmaR.org)
Works with and provides support to the R Foundation and to the key organizations developing, maintaining, distributing and using R software
Over USD $1.4 Million
R Validation Hub, R-Ladies
Stanford Data Institute
R/Medicine, R/Pharma, useR!, LatinR, and more
Guidance on compliant use of R and management of packages
Building a public, validation-ready resource for R packages
Coline Zeballos
Connecting validation experts across the industry
Jaxon Abercrombie, Anuja Das, Antal Martinecz
{riskmetric}Gather and report on risk heuristics to support validation decision-making
Eric Milliman
{riskassessment}A web interface to {riskmetric}, supporting review, annotation and cataloging of decisions
Aaron Clark, Jeff Thompson
Keep your hand raised if…
{riskmetric}, {riskassessment}){riskassessment} App Workstream 5min{riskmetric} workstream 10min.R files through their eSUB portal2.Next date: 🗓️ Aug 20, 2024
Speaker: 👷♂️ Bríd Roberts, Novartis [TODO: how do I change this icon to female?]
Topic:
The Software Open Source (SOS) team manages and executes the risk assessment process for R package validation at Novartis. The team uses an internally developed R package to classify the risk of each package as “low”, “medium”, or “high”.
We analysed the risk assessment data over two time points to determine the impact on the assigned risk categorisation for packages with AND without version changes.
In this talk, we showcase the risk assessments over time, the causes of any risk class changes, and their impact on various teams within our organizations as a result.
Then: Outdated, clumsy, hard to navigate… Insert Image TODO
Now: Update, streamlined, user friendly! Insert Image TODO
{riskassessment} App{riskassessment} AppFunction Explorer page and faster exploration of source codeAbout tabshinymanager deployment{riskassessment} AppTODO: add link and icon for small car
https://app.pharmar.org/riskassessment/
Looking for Volunteer “Leads” to represent
{riskassessment} App{riskassessment} App{riskassessment} App{riskmetric}{riskmetric} RoadmapRunning from risk scores & focusing solely on metric assessments
oyster, srr, pkgstats, etc)Supporting a transparent, open, dynamic, cross-industry approach of establishing and maintaining a repository of R packages.
In close communication with many beloved R projects
pharmaversetargetting repos integration
r-lib/paktargetting pak integration
focus on proving capabilities, quick development
✨ all modelled after r-hub/repos
Define your quality expectations in code
options(available_packages_filters = risk_filter(
# no known security vulnerabilities
quality_cve_count == 0 & (
# package is exceptionally testing
(quality_code_coverage >= 0.8 &
quality_example_coverage >= 0.8 &
quality_r_cmd_check_errors == 0) |
# or is exceptionally well adopted
dplyr::percent_rank(quality_downloads_1yr) > 0.9 |
quality_reverse_dependencies_count >= 10 |
# or seems to follow thorough development practices
(quality_has_website &
quality_vignette_count >= 1 &
quality_author_count >= 3)
)
))*aspirational deviations from proof of concept in github.com/pharmaR/pharmapkgs
Make exceptions explicit
options(available_packages_filters = risk_filter(
exceptions = c("riskmetric", "riskscore"),
# no known security vulnerabilities
quality_cve_count == 0 & (
# package is exceptionally testing
(quality_code_coverage >= 0.8 &
quality_example_coverage >= 0.8 &
quality_r_cmd_check_errors == 0) |
# or is exceptionally well adopted
dplyr::percent_rank(quality_downloads_1yr) > 0.9 |
quality_reverse_dependencies_count >= 10 |
# or seems to follow thorough development practices
(quality_has_website &
quality_vignette_count >= 1 &
quality_author_count >= 3)
)
))*aspirational deviations from proof of concept in github.com/pharmaR/pharmapkgs
Tools for inspecting packages that do not adhere to pre-specified criteria
risk_explain(package = "options")
#> <criteria>
#> * `quality_code_coverage` of `0.43` (< 0.8)
#> * `dplyr::precent_risk(quality_downloads_1yr)` of `0.2` (< 0.9)
#> * `quality_reverse_dependencies_count` of `0` (< 10)
#> * `quality_author_count` of `1` (< 3)*aspirational deviations from proof of concept in github.com/pharmaR/pharmapkgs
Applying filters to a repository mirror
r-hub/reposgithub.com/cran mirrors CRAN packages, hosting source code & binary buildsr-hub/repos indexes packages mirrored on github.com/cran
DownloadUrl field, used by {pak} to fetch packagesPACKAGES file, a standard format for indexing packagespharmaR/repos periodically looks for changes in r-hub/repos
PACKAGES file with additional quality fieldsCalculates package quality metadata on for updated packages and reverse dependencies
Produces logs and necessary reporting metadata
Support in-house packages with open-source tooling
Automating up-to-date quality metrics to support sponsor risk assessment